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Abstract 

Orthogonal Matching Pursuit (OMP) is the canonical greedy algorithm for sparse approxi- 
mation. In this paper we demonstrate that the restricted isometry property (RIP) can be used 
for a very straightforward analysis of OMP. Our main conclusion is that the RIP of order K + 1 
(with isometry constant S < is sufficient for OMP to exactly recover any i\-sparse sig- 

nal. Our analysis relies on simple and intuitive observations about OMP and matrices which 
satisfy the RIP. For restricted classes of if -sparse signals (those that are highly compressible), 
a relaxed bound on the isometry constant is also established. A deeper understanding of OMP 
may benefit the analysis of greedy algorithms in general. To demonstrate this, we also briefly 
revisit the analysis of the Regularized OMP (ROMP) algorithm. 

1 Introduction 

1.1 Orthogonal Matching Pursuit 

Orthogonal Matching Pursuit (OMP) is the canonical greedy algorithm for sparse approximation. 
Letting <3? denote a matrix of size M xN (where typically M < N) and y denote a vector in M. , the 
goal of OMP is to recover a coefficient vector x € l w with roughly K < M nonzero terms so that 
$x equals y exactly or approximately. OMP is frequently used to find sparse representations for 
signals y E M. M in settings where 3> represents an overcomplete dictionary for the signal space [1- 
3]. It is also commonly used in compressive sensing (CS), where y = &x represents compressive 
measurements of a sparse or nearly-sparse signal x G M N to be recovered [4-6]. 

One of the attractive features of OMP is its simplicity. The entire algorithm is specified in 
Algorithm 1, and it requires approximately the same number of lines of code to implement in a 
software package such as Matlab. Despite its simplicity, OMP is empirically competitive in terms 
of approximation performance [3, 7]. 

Theoretical analysis of OMP to date has concentrated primarily on two fronts. The first has 
involved the notion of a coherence parameter [i := maxjj \(fa, <fij)\, where fa denotes column i of the 
matrix When the columns of <3? have unit norm and \i < 2 k~i > ^ nas b een shown [3] that OMP 
will recover any if -sparse signal x from the measurements y = <3?x. This guarantee is deterministic 
and applies to any matrix $ having normalized columns and [i < 2 k-\ ■ 
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Algorithm 1 Orthogonal Matching Pursuit 



input: y, stopping criterion 
initialize: r° = y, x° = 0, A = 0, £ = 
while not converged do 
match: // = $ T r^ 

identify: A £+1 = A e U {arg maxj \h (if multiple maxima exist, choose only one) 

update: x e+1 = argmin 2 . supp ( 2 )c A f+i \\y ~ ®z\\ 2 

r e+1 = y — &x i+1 

£ = £ + 1 

end while 

output: x = x e = argmin z . supp ( z )cA^lb - ®z\\ 2 

The second analytical front has involved the notion of probability. Suppose x G R N with 
1 1 a? 1 1 o := l su PP( x )l < K and that $ is drawn from a suitable random distribution (independently 
of x) with M = 0(K log(N)) rows. Then with high probability, OMP will recover x exactly from 
the measurements y = &x [6]. It is not guaranteed, however, that any such fixed matrix will allow 
recovery of all sparse x simultaneously. 

1.2 The Restricted Isometry Property 

As an alternative to coherence and to probabilistic analysis, a large number of algorithms within the 
broader field of CS have been studied using the restricted isometry property (RIP) for the matrix 
$ [8]. A matrix <I> satisfies the RIP of order K if there exists a constant 5 € (0, 1) such that 

(l-«5)||x|||<||$ : r|||<(l + «5)||x||2 (1) 

holds for all x such that ||a;||o < K. In other words, <I> acts as an approximate isometry on the 
set of vectors that are i^-sparse. Much is known about finding matrices that satisfy the RIP. For 
example, if we draw a random M x N matrix $ whose entries faj are independent and identically 
distributed sub-Gaussian random variables, then provided that 

fK\og(N/K)\ 

m=o ( v — V^J' (2) 

with high probability $ will satisfy the RIP of order K [9, 10]. 

When it is satisfied, the RIP for a matrix $ provides a sufficient condition to guarantee successful 
sparse recovery using a wide variety of algorithms [8, 11-19]. As an example, the RIP of order 
IK (with isometry constant 5 < \[2 — 1) is a sufficient condition to permit £i-minimization (the 
canonical convex optimization problem for sparse approximation) to exactly recover any K-sparse 
signal and to approximately recover those that are nearly sparse [11]. The same RIP assumption 
is also a sufficient condition for robust recovery in noise using a modified £i-minimization [11]. 

Despite the considerable attention that has been paid to both OMP and the RIP, analysis 
of OMP using the RIP has been relatively elusive to date. However, several alternative greedy 
algorithms have been proposed — all essentially modifications of OMP — that are apparently much 
more amenable to RIP-based analysis. The Regularized Orthogonal Matching Pursuit (ROMP) [13, 
14] and Subspace Pursuit (SP) [16] algorithms differ from OMP in the identification step, while the 
Compressive Sampling Matching Pursuit (CoSaMP) [15] and DThresh [17] algorithms differ from 
OMP in both the identification and the update steps. For each of these algorithms it has been 
shown that the RIP of order CK (where C > 2 is a constant depending on the algorithm) with 5 
adequately small is sufficient for exact recovery of K sparse signals. 
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1.3 Contributions 



Our contributions in this paper are twofold. First, we begin in Section 2 with some very simple 
observations regarding OMP. Many of these facts are known to practitioners in the field but may 
not be obvious to a novice, and we feel that such readers may find value in a short exposition. 

Critically, these observations also set the stage for our main results in Section 3, in which we 
demonstrate that the RIP can be used for a very straightforward analysis of OMP. Our analysis 
revolves around three key facts: (1) that in each step of the algorithm, the residual vector r f ~ can 
be written as a matrix times a sparse signal, (2) that this matrix satisfies the RIP, and (3) that 
consequently a sharp bound can be established for the vector h of inner products. Our main 
conclusion, Theorem 3.1, states that the RIP of order K + 1 (with 5 < tt^O is sufficient for OMP 
to exactly recover any K-sparse signal in exactly K iterations. However, for restricted classes of 
-fT-sparse signals (those with sufficiently strong decay in the nonzero coefficients), a relaxed bound 
on the isometry constant can be used. We discuss such extensions of our results in Section 4. A 
deeper understanding of OMP may also benefit the analysis of greedy algorithms in general. To 
demonstrate this, we briefly revisit the analysis of the ROMP algorithm in Section 4. 

1.4 Context 

Let us place Theorem 3.1 in the context of the OMP literature. Using the RIP as a sufficient 
condition to guarantee OMP performance is apparently novel. Moreover, the fact that our bound 
requires only the RIP of order K + 1 is apparently unique among the published CS literature; much 
more common are results requiring the RIP of order 1.75K [12], 2K [11, 13], 3K [16, 18], 4K [15], 
and so on. Of course, such results often permit the isometry constant to be much larger. 1 

If one wishes to use the RIP of order K + 1 as a sufficient condition for exact recovery of all 
if-sparse signals via OMP (as we have), then little improvement is possible in relaxing the isometry 
constant 5 above 7^7^- In particular, there exists a matrix satisfying the RIP of order K + 1 with 

5 < ^7= for which there exists a i^T-sparse signal x G R N that cannot be recovered exactly via K 
iterations of OMP. (This is conjectured in [16] with a suggestion for constructing such a matrix, 
and for the case K = 2 we have confirmed this via experimentation.) 

Unfortunately, from (2) we see that finding a matrix <3> satisfying the RIP of order K + 1 with 
an isometry constant 5 < may require M = 0(K 2 \og(N/K)) random measurements. If 

one wishes to guarantee exact recovery of all K-sparse signals via OMP (as we have), then little 
improvement is possible in relaxing this number. In particular, it has been argued [20] that when 
M < K 3 / 2 , for most random M x N matrices $ there will exist some if-sparse signal x £ M N that 
cannot be recovered exactly via K iterations of OMP. 

It is also worth comparing our RIP-based analysis with coherence-based analysis [3], as both 
techniques provide a sufficient condition for OMP to recover all -fT-sparse signals. It has been 
shown [6] that in a random M x N matrix, the coherence parameter fj, is unlikely to be smaller 
than log(iV)/\/M. Thus, to ensure [i < 2 k-\ i one requires M = 0(K 2 log 2 (iV)), which is roughly 
the same as what is required by our analysis. We note that neither result is strictly stronger 
than the other; we have confirmed experimentally that there exist matrices that satisfy our RIP 
condition but not the coherence condition, and vice versa. 

Finally, we note that the aforementioned modifications of OMP (the ROMP, SP, CoSaMP, and 
DThresh algorithms) all have RIP-based guarantees of robust recovery in noise and stable recovery 

x Note that a smaller order of the RIP is not necessarily a weaker requirement if the required constant is also 
significantly smaller. For example, Corollary 3.4 of [15] implies that if $ satisfies the RIP of order K + 1 with 
constant 5, then $ also satisfies the RIP of order 2K with constant 48. 
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of non-sparse signals. To date, no such RIP-based or coherence-based guarantees have been provided 
for OMP itself. We speculate that our perspective may help to further the understanding of OMP 
and perhaps provide a route to such a guarantee. At present, however, this remains a topic of 
ongoing work. 

1.5 Notation 

Before proceeding, we set our notation. Suppose A C {1, 2, ... , N}. We let A c = {1, 2, . . . , N}\A. 
By x|a we mean the length |A| vector containing the entries of x indexed by A. 

By <E>a we mean the M x |A| matrix obtained by selecting the columns of <!> indexed by A, and 
by 7£(<I>a) we mean the range, or column space, of <E>a. We will assume throughout that when 
|A| < M, <E>a is full rank, in which case we let <3? A := (<& a $a) _1 ^a denote the Moore-Penrose 
pseudoinverse of <&a- 

We denote the orthogonal projection operator onto ^(^a) by Pa := ^a^A' Similarly, P A := 
(/ — Pa) is the orthogonal projection operator onto the orthogonal complement of 1Z(&a). We note 
that any orthogonal projection operator P obeys P = P T = P 2 . 

Finally, we define A a := P\®- This matrix is the result of orthogonalizing the columns of 
against TZ(&\). It is therefore equal to zero on columns indexed by A. 

2 Observations 

Let us begin with some very simple observations regarding OMP as presented in Algorithm 1. 
The key idea is to try to iteratively estimate a set A that contains the locations of the nonzeros 
of x by starting with A = and then adding a new element to A in each iteration. In order 
to select which element to add, the algorithm also maintains a residual vector r ^ 7£($a) that 
represents the component of the measurement vector y that cannot be explained by the columns 
of $a- Specifically, at the beginning of the I th iteration, A e is our current estimate of supp(x), and 
the residual r £ is defined as r e = y — &x e where supp(ar) C A e . The element added to A e is the 
index of the column of <3? that has the largest inner product with r l . 

Our first observation is that can be viewed as the orthogonalization of y against the previously 
chosen columns of <I>. To see this, note that the solution to the least squares problem in the update 
step is given by 

x e \ A e = $\ iy and x £ | (A<)c = 0. (3) 

Thus we observe that 

r l = y - $x e = y - & A t$\ty = (!- P m)v = P hv- 

Note that it is not actually necessary to explicitly compute x l in order to calculate r l . 

Our second observation is that, in the matching step, one may correlate either with the 
columns of or with the columns of A A e. To see this equivalence, observe that r e = P^eU = 

h' = $V = ^{P^fP^y = A T Ae r e . (4) 
Incidentally, along these same lines we observe that 
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From this we note that it is not actually necessary to explicitly compute r in order to calculate 
the inner products during the matching step; in fact, the original formulation of OMP was stated 
with instructions to orthogonalize the remaining columns of <I> against those previously chosen 
and merely correlate the resulting vectors against y [1,2]. Additionally, we recall that, in A A e, all 
columns indexed by A e will be zero. It follows that 

h e (j)=0 Vj G A e , (5) 

and so, since A e = A e_1 U {j*} with j* £ A* -1 , 

|A'| = t. (6) 

Our third observation is that, in the case of noise-free measurements y = $x, we may write 

/ = P^y = P^x = A A ix. 

Again recalling that all columns of A A e indexed by A e are zero, we thus note that when supp(x) C 
A e , r e = 0, and from (3) we also know that x e = x exactly. It will also be useful to note that for 
the same reason, we can also write 

r e = A A ex\ (7) 

where 

x e \ A e = and x e \/^e\ c = x\t A ey. (8) 



3 Analysis 

Our analysis of OMP will center on the vector h . In light of (4) and (7), we see that A A t plays a 
role both in constructing and in analyzing the residual vector. In Lemma 3.2 below, we show that 
the matrix A A e satisfies a modified version of the RIP. This allows us to very precisely bound the 
values of the inner products in the vector h*. 

We begin with two elementary lemmas whose proofs are given in the Appendix. Our first 
result, which is a straightforward generalization of Lemma 2.1 of [11], states that RIP operators 
approximately preserve inner products between sparse vectors. 

Lemma 3.1 Let u,v G 1^ be given, and suppose that a matrix ^ satisfies the RIP of order 
max(||n + v\\o, \\u — v\\o) with isometry constant 5. Then 

\($u,Vv) - (u,v)\ < S\\u\\ 2 \\v || 2 . (9) 

One consequence of this result is that sparse vectors that are orthogonal in remain nearly 
orthogonal after the application of *S>. From this observation, it was demonstrated independently 
in [21] and [16] that if $ has the RIP, then A A satisfies a modified version of the RIP. 

Lemma 3.2 Suppose that <I> satisfies the RIP of order K with isometry constant 5, and let A C 
{1,2,... , N}. If\A\ < K then 

(i - IMIl < Pa«||1 < (i + (io) 

for all u e R N such that \\u\\q < K — \A\ and supp(w) n A = 0. 
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In other words, if satisfies the RIP of order K, then A A acts as an approximate isometry on 
every (K— |A|)-sparse vector supported on A c . From (7), we recall that the residual vector in OMP 
is formed by applying A A e to a sparse vector supported on (A e ) c . Combining the above results, 
then, we may bound the inner products h l {j) as follows. 

Lemma 3.3 Let A C {1, 2, ... , N} and suppose x G 1^ with supp(2?) n A = 0. Define 

h = A T A A A x. (11) 
Then if satisfies the RIP of order \\x\\o + |A| + 1 with isometry constant 5, we have 

\h{j)-x{j)\<^- 8 \\x\\2 (12) 

for all j A. 

Proof: From Lemma 3.2 we have that the restriction of A A to the columns indexed by A c 
satisfies the RIP of order (||x||o + |A| + 1) — |A| = ||x||o + 1 with isometry constant 5/(1 — S). By 
the definition of h, we also know that 

h(j) = (A A x,A A ej), 

where ej denotes the j th vector from the cardinal basis. Now, suppose j ^ A. Then because 
\\x ± e j- 1 1 o < 1 1 5? 1 1 o + 1 and supp(x ± ej) n A = 0, we conclude from Lemma 3.1 that 

\h{j) = \(A A x,A Aej ) - (x,ej)\ < ^— ^||x|| 2 || ej -||2. 

Noting that ||ej||2 = 1, we reach the desired conclusion. □ 

With this bound on the inner products h^(j), we may derive a sufficient condition under which 
the identification step of OMP will succeed. 

Corollary 3.1 Suppose that A, <1>, x meet the assumptions specified in Lemma 3.3, and let h be as 
defined in (11). If 

25 , , 

F oo > i ? F b, (13) 

l — o 

we are guaranteed that argmax^ \h(j)\ £ supp(x). 

Proof: If (12) is satisfied, then for indices j ^ supp(x), we will have \h(j)\ < jz^\\x\\2- 
(Recall from (5) that h(j) = for j G A.) If (13) is satisfied, then there exists some j £ supp(2?) 
with \x(j)\ > yz$\\%\\2- From (12) and the triangle inequality, we conclude that for this index j, 
\h(j)\ > t^P|| 2 . □ 

By choosing 5 small enough, it is possible to guarantee that the condition (13) is satisfied. In 
particular, the lemma below follows from standard arguments. 



Lemma 3.4 For any u G 1^, ||u||oo > IM| 2 /\/|Mlo- 

Putting these results together, we can now establish our main theorem concerning OMP. 

Theorem 3.1 Suppose that $ satisfies the RIP of order K + 1 with isometry constant 5 < 
Then for any x G 1^ with \\x\\o < K, OMP will recover x exactly from y = <3?x in K iterations. 
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Proof: The proof works by induction. We start with the first iteration where h° = & T &x and 
note that = Aq. Because ||x||o < K, Lemma 3.4 states that ||x||oo > 77x - One can a ^ so check 
that 5 < implies that < -j=. Therefore, we are guaranteed that (13) is satisfied, and so 

from Corollary 3.1 we conclude that argmaxj \h°(j)\ £ supp(x). 

We now consider the general induction step. Suppose that we are at iteration £ and that all 
previous iterations have succeeded, by which we mean that A e C supp(x). From (8), we know that 
supp(x^) n A e = and that ||5^||o < K — £. From (6), we know that |A^| = I. By assumption, $ 
satisfies the RIP of order K + 1 = (K - I) + I + 1 > \x% + \A e \ + 1. Finally, using Lemma 3.4, 
we have that ____ 

u~£u ^ \\ xe h ^ \\x e h ^ 2 $ u~eu 

From Corollary 3.1 we conclude that argmaxj \h e (j)\ G supp(x^) and hence A e+1 C supp(x). □ 



4 Extensions 

4.1 Strongly-decaying sparse signals 

For even moderate values of the isometry constant 5 there exist sparse signals that we can ensure 
are recovered exactly. For example, if the decay of coefficients is sufficiently strong in a sparse 
signal, we may use Lemma 3.3 to ensure that the signal entries are recovered in the order of their 
magnitude. 

For any x 6 M. N with ||x||o < K we denote by x'(j) the entries of x ordered by magnitude, i.e., 

|x'(l)| > \x'{2)\ > ■ ■ ■> \x'(K)\ > 
with x'(K + 1) = x'(K + 2) = • • • = x'(N) = 0. 

Theorem 4.1 Suppose that <3? satisfies the RIP of order K + 1 with isometry constant 8 < |. 
Suppose x € 1^ with \\x\\q < K and that for all j G {1, 2, . . . , K — 1}, 

\Aj)\ >a . 



1^0' + i)l 

" > 1 - 2-L- ' (14) 
i/ien OMP mZZ recover x exactly from y = <I>x in K iterations. 

Proof: The proof again proceeds by induction. At each stage, OMP will choose the largest 
entry of x e . To see this, note that by (12) we have \h l {j) — x e (j)\ < jz^\\x^\\2- The nonzero entries 
of X s - will be comprised of x'(£ + l),x'(£ + 2), . . . , x'(K). Thus, 

,*n, < J W(t + up + (k - ^EgpF = + m V a , + - 1) < i^ +1 V vi^t). 

V or a a 

Now, for the largest entry of x , we have 

\h\j)\ > \x'{£ + 1)| - ^'^^V + V^l) = ^^(« - ^(a + v^T)) (15) 
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while for all other entries we have 

\h\j)\ < \x'{l + 2)| + / - |g/( * +1)l (a + v^T) < + + -^-(a + y/K^T)). (16) 
1 — o a a l — o 

From (14), it follows that (15) is greater than (16). □ 
4.2 Analysis of other orthogonal greedy algorithms 

We now demonstrate that the techniques used above can also be used to analyze other orthogonal 
greedy algorithms. We focus on ROMP [13, 14] for the purpose of illustration, but similar methods 
should be able to simplify the analysis of other orthogonal greedy algorithms such as SP [16]. 2 

We first briefly describe the difference between ROMP and OMP, which lies only in the iden- 
tification step: whereas OMP adds only one index to A e at each iteration, ROMP adds up to K 
indices to at each iteration. Specifically, ROMP first selects the indices corresponding to the K 
largest elements in magnitude of h (or all nonzero elements of h if h has fewer than K nonzeros) , 
and denotes this set as fi . The next step is to regularize this set so that the values are comparable 
in magnitude. To do this, define R(Q e ) := {l)CSl f : \h e (i)\ < 2\h e (j)\ Vi, j G SI}, and set 

Sl e := axgmax neR( p')\\ h %h, 

i.e., is the set with maximal energy among all regularized subsets of Q*. Finally, setting A i+1 = 
A U Oq, the remainder of the ROMP algorithm is identical to OMP. 

In order to analyze ROMP, we will need only two preliminary lemmas from [13], which we state 
without proof. Note that Lemma 4.1, which is essentially a generalization of Lemma 3.3, is stated 
using slightly weaker assumptions than those stated in [13]. The present version can easily be 
obtained using the same proof. 

Lemma 4.1 ((1) in Prop. 3.2 of [13]) Let r C {1, 2, ... , N} and x G R N be given. Then if * 
satisfies the RIP of order |supp(x) U T| with isometry constant 5, we have 

||(* T *z)|r - x| r || 2 < S\\x\\ 2 . 

Lemma 4.2 (Lemma 3.7 of [13]) Let u G R K , K > 1, be arbitrary. Then there exists a subset 
r C {1, . . . , K} such that \u(i)\ < 2\u(j)\ for all i,j G T and 

1 

Mr 2 > . Mb- 

2.5-y/log 2 K 

Using these lemmas, we now provide a simplified proof of the main result of [13] concerning the 
recovery of sparse signals using ROMP. 3 

Theorem 4.2 Suppose that $ satisfies the RIP of order 3K with isometry constant 5 < 0.13/ -y/log 2 K. 
Then for any x G R N with ||x||o < K, ROMP will recover x exactly from y = <&x in at most K 
iterations. 

2 Some of the greedy algorithms that have been proposed recently, such as CoSaMP [15] and DThresh [17], do 
not orthogonalize the residual against the previously chosen columns at each iteration, and so the techniques above 
cannot be directly applied to these algorithms. However, this orthogonalization step could easily be added (which in 
the case of CoSaMP yields an algorithm nearly identical to SP). Orthogonalized versions of these algorithms could 
then be studied using these techniques. 

3 Note that we assume that <!? satisfies the RIP of order 3K with constant S < 0.13/ \/log 2 K. Using Corollary 3.4 
of [15], we can replace this with the assumption that $ satisfies the RIP of order 2K with constant 8 < .043/\/log 2 K. 
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Proof: The proof works by showing that at each iteration, 



|^ nsu PP (x)| > ^\n e \. (17) 

If (17) is satisfied for 0, 1, ...,£ — 1, then at iteration £ we have that 

|A £ nsupp(x)|>i|A £ |. (18) 

It follows that, before |A^| exceeds 2K, we will have supp(x) C A . Because $ satisfies the RIP of 
order 3K > 2K, at termination, will be full rank. From (3) we conclude that x = x exactly. 

To prove (17), we again proceed by induction. Hence, we assume that (17) holds for 0, 1, ... , £—1, 
and thus (18) holds for iteration £. We next assume for the sake of a contradiction that (17) does 
not hold for iteration £, i.e., that 

|fig\supp(x)| > \\^\. (19) 

Define the sets T = S7q \ supp(x) and S = supp(x) \ A e = supp(a^), where x e is defined as in (8). 
Recall that we can write = A^A^ex^ . Thus, using the assumption that \T\ > ^\£Iq\ and the 
facts that TCfij and Q e Q £ i?(ft £ ), one can show that 

||^| T || 2 >-L||^| n ,|| 2 . (20) 

We now observe that 

\\h% () h>—^—\\h e \ Qe h, (21) 
2.5 v / log 2 K 

which follows from Lemma 4.2 and the fact that VLq is the maximal regularizing set. From the 
maximality of Q e and the fact that |»S| < K, we have that \\h \ne\\ 2 > ||^ |s||2 5 so that by combining 
(20) and (21) we obtain 

\\h e \ T \\ 2 > . 1 \\h e \sh- (22) 

2.5V51og 2 K 

Note that |5Usupp(x^)| = \S\ < K and since |A^| < 2K, from Lemma 3.2 we have that A A e satisfies 
the RIP of order at least K with constant 5/(1 — 5), thus Lemma 4.1 implies that 



\h e \s ~ x e \sh < - ? \\x e 



5 

'"2- 



1 - 6' 

Since x e \ s = x e , \\h e \ s - x e \sh = \\x e - h e \ s \\ 2 > \\x e \\ 2 - ll^lslh, and thus 



iiJi n \ 1 ~ 

\h l \sh > -, f\\ x 2- 



Hence, 



|| ft '| T fc > ^-^^WA, (23) 
2.5^5 bg 2 A" 

On the other hand, since |supp(x^)| + \A l n supp(x)| = K, from (18) we obtain that |supp(2^)| < 
K - \\A% Thus, \T U supp(x^)| < \T\ + |supp(^)| < 2K - ±\A e \. Furthermore, A A i satisfies the 
RIP of order 3K - \A e \ = 3K - ±\A e \ - ±\A e \. Since |A £ | < 2K, we have that A M satisfies the RIP 
of order at least 2K — \\A i \ with constant 6/(1 — 5). Thus, Lemma 4.1 also implies that 

\\h e \ T \\ 2 = \\h e \ T - x e \ T \\ 2 < -*—\\tf\\ 2 . (24) 
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This is a contradiction whenever the right-hand-side of (23) is greater than the right-hand-side of 
(24), which occurs when 5 < 1/(2 + 2.5-^/5 log 2 K). Since log 2 K > 1, we can replace this with the 
slightly stricter condition 6 < l/((2 + 2.5^5) ^log 2 K) « 0.1317/y^Iog^ if. □ 

Observe that when K = 1, this proof (as well as the proofs in [13, 14]) break down since 
Lemma 4.2 does not apply. However, when K = 1 the ROMP algorithm simply reduces to OMP. 
In this case we can apply Theorem 3.1 to verify that ROMP succeeds when K = 1 provided that 
<3? satisfies the RIP of order 2 with isometry constant 5 < 1/3. 



Appendix 

Proof of Lemma 3.1: We first assume that ||u||2 = IMI2 = 1- From the fact that 

||u±u||l = IMI2 + ||u||!±2(u,t;) =2±2(u,v) 
and since ^ satisfies the RIP, we have that 

(l-<S)(2±2(u,u)) < ||¥u±¥i;||| < (l + 5)(2±2{u,v)). 
From the parallelogram identity we obtain 

it T v 1 / 1 1 t t ll2 llT T „2\ ^ (1 + (u,v))(l +6) - (1 - (u,v))(l - 5) . . . 
tfu) = - {\\Vu + *w||2 - - VvWZ) < — — — — — = (u, v) + (5. 

Similarly, one can show that (\Fu, Vl'i;) > (u,v) — 5, and thus \(^u,^v) — (u,v)\ < S. The result 
follows for u, v with arbitrary norm from the bilinearity of the inner product. □ 

Proof of Lemma 3.2: From the definition of A A we may decompose A A u as A A u = <&u — P A Qu. 
Since Pa is an orthogonal projection, we can write 

\\$ug = \\P A <S>u\\l + \\A A u\\l (25) 

Our goal is to show that ||<&m||2 ~ HAv^lh, or equivalently, that ||Pa$m||2 is small. Towards this 
end, we note that since P A &u is orthogonal to A A u, 

{P A <S>u, $u) = <P A *«, Pa$u + A A u) = (P A $u, P A <S>u) + {P A ®u, A A u) = ||Pa*«||1- (26) 

Since Pa is a projection onto 1Z(Q A ) there exists az£ with supp(z) C A such that P A &u = &z. 
Furthermore, by assumption, supp(ii) fl A = 0. Hence (u, z) = and from the RIP and Lemma 3.1, 

\{P A <f>u,®u)\ \($z,$u)\ \(<f>z,$u)\ S 



||Pa*«||2||*«||2 ||^||2||$n||2 (1 — <5) 1 1 ^ 1 1 2 1 1 ^ 1 1 2 
Combining this with (26), we obtain 

\\Pa$u 2 < z f \\® u 2- 

1 — o 

Since we trivially have that ||Pa$u||2 > 0, we can combine this with (25) to obtain 

2\ 

2 ^ II A . „. I|2 ^ 11 jf.„. 11 2 



Since ||u||o < K, we can use the RIP to obtain 




!-(t^t) |(l-«<MA«||i<(l + «)W, 



which simplifies to (10). □ 
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